Signature-based unique identifier

ABSTRACT

The technology described herein generates a unique identifier for a visual media that comprises pre-printed visual indications on the visual media and a user&#39;s handwritten signature. The location of the signature on the visual media can be determined by including preprinted fiducial marks on the visual media. The fiducial markers act as landmarks that allow the size and location of the signature to be determined in absolute terms. The unique identifier is then stored in computer memory on a user-experience server. The user-experience server can associate the unique identifier with a digital asset, such as an image or video, designated by the user. When the unique identifier is provided to the user-experience server a second time, the digital asset can be retrieved and output to the computing device that provided the unique identifier.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Aspects of the technology described herein generate a unique identifierfor a visual media that comprises pre-printed visual indications on thevisual media and a user's handwritten marking, such as a signature. Thevisual media includes printed media, such as a greeting card, gift card,or ornament.

The unique identifier is unique because of visual characteristicscreated by a user's handwritten signature or other marking incombination with the location of the marking on the visual media. Thelocation of the handwritten marking on the visual media can bedetermined by including preprinted fiducial marks on the visual media.In general, it is beneficial for a fiducial marker to be a uniquepattern that is easy for the computer software to recognize as afiducial marker.

The fiducial markers act as landmarks that allow the size and locationof the handwritten marking to be determined in absolute terms. Theabsolute size and location of the handwritten marking in combinationwith the handwritten marking itself can be used to create a uniqueidentifier that in turn identifies the visual media to which thehandwritten marking is affixed. In one aspect, a QR code or othermachine-readable indicia may be provided on the visual media to identifya stock-keeping unit (“SKU”) for the visual media, such as the greetingcard.

The machine-readable indicia may be used in combination with thehandwritten marking and fiducial marks to create the unique identifier.

The unique identifier is then stored in computer memory on auser-experience server. The user-experience server can associate theunique identifier with a digital asset, such as an image or video,designated by the user. The visual media can then be communicated to arecipient. For example, a greeting card can be mailed from a sender to arecipient.

When the unique identifier is provided to the user-experience server asecond time by a second computing device, the digital asset can beretrieved and output to the second computing device. For example, arecipient of a greeting card could scan the card to extract the uniqueidentifier and then retrieve an associated digital asset, such as apersonalized greeting video. Each time the unique identifier isprovided, the digital asset can be output. In one aspect, storing theunique identifier includes storing an image of the handwritten markingalong with size and location characteristics ascertained from ananalysis of the handwritten marking in relation to the fiducial markers.Supplemental information may be stored with the unique identifier, suchas a SKU for the visual media, sender information (e.g., a name or userID), a time the unique identifier was created, and the like.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is described in detail below with reference tothe attached drawing figures, wherein:

FIG. 1 is a diagram of a user signing a visual media, in accordance withaspects of the technology described herein;

FIG. 2 is a diagram of a sender capturing an image of the sender'ssignature on the visual media, in accordance with aspects of thetechnology described herein;

FIG. 3 is a diagram of a digital asset selection interface that a senderuses to associate a digital asset with the visual media, in accordancewith aspects of the technology described herein;

FIG. 4 is a diagram of a recipient capturing an image of a sender'ssignature on the visual media, in accordance with aspects of thetechnology described herein;

FIG. 5 is a diagram of a recipient viewing the digital asset through therecipient's user device, in accordance with aspects of the technologydescribed herein;

FIG. 6 is a block diagram of an exemplary computing environment suitablefor use in implementing embodiments of the present disclosure;

FIG. 7 is a block diagram illustrating an exemplary media-identificationsystem in which some embodiments of the present disclosure may beemployed;

FIG. 8 is a flow diagram that illustrates method for uniquelyidentifying a visual media, in accordance with aspects of the technologydescribed herein;

FIG. 9 is a flow diagram that illustrates method for uniquelyidentifying a visual media, in accordance with aspects of the technologydescribed herein;

FIG. 10 is a flow diagram that illustrates method for uniquelyidentifying a visual media, in accordance with aspects of the technologydescribed herein; and

FIG. 11 is a block diagram that illustrates an exemplary computingdevice.

DETAILED DESCRIPTION

The subject matter of the present disclosure is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described. Each method described herein may comprisea computing process that may be performed using any combination ofhardware, firmware, and/or software. For instance, various functions maybe carried out by a processor executing instructions stored in memory.The methods may also be embodied as computer-usable instructions storedon computer storage media. The methods may be provided by a standaloneapplication, a service or hosted service (standalone or in combinationwith another hosted service), or a plug-in to another product, to name afew.

Aspects of the technology described herein generate a unique identifierfor a visual media that comprises pre-printed visual indications on thevisual media and a user's handwritten marking, such as a signature,doodle, note, or drawing. The visual media includes printed media, suchas a greeting card, gift card, post card, picture frame, Christmasornament, or the like. The visual media can also include electronicmedia, such as an e-card, email, social media post, and the like.

The unique identifier is unique because of visual characteristicscreated by a user's handwritten marking in combination with the locationof the marking on the visual media. In one aspect, the marking is ahandwritten signature. In general, people have unique signatures whenthey are handwritten. For example, it would be rare for two people withthe same name to have visually identical signatures. That is especiallytrue because the signature is being analyzed by a computer imagingsystem, which can detect fine distinctions. As used herein, a signatureis a visible production of all or part of a user's name by hand using apen, pencil, stylus, finger, or other writing implement. The signaturecould be in cursive or printed letters. The signature can also compriseentirely a mark or marks the user commonly uses to identify himself orherself.

The location of the marking, such as a signature, on the visual mediacan be determined by including preprinted fiducial marks on the visualmedia. In general, it is beneficial for a fiducial marker to be a uniquepattern that is easy for the computer software to recognize as afiducial marker. It is also beneficial for the marker to beasymmetrical. It is easier to determine the relative positioning of thecamera and the marker if some aspect of the marker is asymmetrical. Anasymmetrical marker makes it possible for the vision software todetermine which way the marker is oriented.

The fiducial markers act as landmarks that allow the size and locationof the marking to be determined in absolute terms. The absolute size andlocation of the marking in combination with the handwritten markingitself can be used to create a unique identifier that in turn identifiesthe visual media to which the signature is affixed. In one aspect, a QRcode or other machine-readable indicia may be provided on the visualmedia to identify a SKU for the visual media, such as the greeting card.This machine-readable indicia may be used in combination with thehandwritten marking to create the unique identifier.

As used herein, the fiducial marks can be stand-alone marks, integratedmarks, or derived from the actual design on the visual media (i.e.,natural feature tracking). The stand-alone marks are visible apart froma design on the visual media. Stand-alone marks are illustrated withreference to FIG. 1. The examples herein are described in terms ofstand-alone fiducial marks, but these examples are not intended to belimiting. Any type of fiducial mark or mark serving the purposes of thefiducial mark described herein, regardless of designation, may be used.

The fiducial marks can also be integrated. Integrated fiducial marks mayalso be described as aesthetic marks. Integrated fiducial marks areintegrated into a design or possibly designed around. The typical userwould not recognize the integrated fiducial marks as anything other thanthe design because they are often just shapes commonly seen in differentdesigns. For example, the integrated fiducial marks can be a pair ofcircles, ovals, or other shapes. The design development may be performedwith any color, texture, shape or other features as long as the circle(or other shape) pairs are integrated. The ratio of the radius of twocircles in a pair can uniquely identify an integrated mark from a lookup table. Other characteristics of the mark and associated visual media,if applicable, could be extracted from the look up table and used toperform operations described herein. For example, spatial informationcould be retrieved from the look up table and used to identifycharacteristics of a signature or other user mark on a visual media.

In one aspect, the fiducial marks use natural feature tracking. Naturalfeature tracking evaluates a design with machine vision technology andidentifies key points within a design that can serve as fiducial marks.For example, the features could include corners, edges, blobs, withoutusing specifically designed ID markers. These features allow the systemto identify the specific design, along with the spatial features of thevisual media on which the design is found. Understanding the spatialfeatures of the design allow for the spatial features of a signature orother mark to be identified, as described subsequently.

The unique identifier is then stored in computer memory on auser-experience server. The user-experience server can associate theunique identifier with a digital asset designated by the user. When theunique identifier is provided to the user-experience server a secondtime, the digital asset can be output to the computing device thatprovided the unique identifier the second time. For example, a recipientof a greeting card could scan the card to extract the unique identifierand then retrieve an associated digital asset, such as a personalizedgreeting video. Each time the unique identifier is provided, the digitalasset can be output. In one aspect, storing the unique identifierincludes storing an image of the handwritten marking along with othersize and location characteristics ascertained from an analysis of thehandwritten marking in relation to the fiducial markers. Supplementalinformation may be stored with the unique identifier, such as a SKU forthe visual media, sender information (e.g., a name or user ID), a timethe unique identifier was created, and the like.

The technology allows a sender of the visual media to associate theunique identifier with a digital asset, such a picture, audio file, orvideo file. The sender can then communicate the visual media to arecipient. The recipient uses an application to retrieve the digitalasset associated with the unique identifier. The digital asset can thenbe output for the recipient's enjoyment.

In the case of the printed media, a camera on a computing device, suchas smart phone, can be used by the sender to capture and store theunique identifier. Similarly, the recipient can use a smart phone tocapture an image of the unique identifier. The image may then beprocessed by an application on the smart phone and used to retrieve thedigital asset associated with the unique identifier. Alternatively, thephone can send the image to a cloud-based service that analyzes theimage and retrieves the digital asset, which is the communicated to thesmart phone.

FIGS. 1-5 illustrate the use of the technology with a greeting card.However, aspects of the technology described herein are not limited touse with a greeting card. The example of FIGS. 1-5 describes thetransfer of a greeting card between a first user and a second user. Thefirst user is described as the sender and the second user is describedas the recipient. In addition, FIGS. 1-5 describe the handwrittenmarking as a signature. Use of other handwritten markings is consistentwith aspects of the technology described herein.

Turning now to FIG. 1, a sender 114 first applies his or her signature110 on the physical greeting card 100, such as with a pen 112. Thegreeting card includes four fiducial markers. The first fiducial marker101 is located in the upper left-hand corner of the greeting card 100.The second fiducial marker 102 is located in the lower left-hand cornerof the greeting card 100. The third fiducial marker 103 is located inthe upper right-hand corner of the greeting card 100. The fourthfiducial marker 104 is located in the lower right-hand corner of thegreeting card 100. While the fiducial markers included here arestand-alone markers, integrated, or natural markers could also be used.The greeting card 100 also includes stock-keeping unit (“SKU”) 105. TheSKU identifies the greeting card model, but not the individual greetingcard. In other words, all greeting cards of the same model may have thesame SKU. The SKU may be used to identify a fiducial mark patternassociated with the card model. Different card models may includedifferent fiducial marks, different arrangements of fiducial marks, anddifferent amounts of fiducial marks, among other differences.

Turning now to FIG. 2, the sender opens a media-identification interfaceof a greeting card application running on a mobile device 120 and takesa picture of the signature area with the mobile-device's camera. Themedia-identification interface may provide instructions that ask theuser to capture the portion of the greeting card including the signatureand the minimum required amount of fiducial marks. In other words, theportion of the visual media captured by the camera should include allfiducial marks required to generate a unique identifier that includesthe signature 110. For example, it may be sufficient to include only thesecond fiducial marker 102 and the fourth fiducial marker 104 in theimage in order to create a unique identifier. However, when the SKU isincluded on the visual media the instruction may request that the SKUalso be included in the image. In this case, instructions could indicatethat the signature should be captured along with at least two fiducialmarks and the SKU. It should be pointed out that FIG. 2 only includes asingle signature 110, however, aspects of the technology describedherein can work with multiple signatures and use multiple signatures togenerate the unique identifier.

The media-identification interface and/or greeting card application canperform a quality check on the image. If all required fiducial marks andthe SKU watermark are not within the captured image, then the sender maybe prompted to capture a second image. Alternatively or additionally,the quality check can be performed in real time through analysis of thevideo being received through the camera and instructions provided toadjust the camera location as needed. If all fiducial marks and SKUwatermark are within the field of view, then the greeting cardapplication instructs the user to capture an image, for example bytapping the screen. The greeting card application then generates aunique identifier using the signature, one or more fiducial marks, andoptionally the SKU or information derived from the SKU. The forms theunique identifier can take will be described in more detail withreference to FIG. 7.

Turning now to FIG. 3, the sender 114 selects one or more digital assetsto associate with the greeting card 100. The digital assets may beidentified through a media-identification interface provided by thegreeting card application. The digital assets can include assets locatedon the mobile device, such as existing pictures and videos. The digitalassets can include assets located on a server, such as publicly orprivately available pictures and videos. The digital asset may also becreated by the media-identification interface. For example, themedia-identification interface can facilitate the user recording anaudio or video message that becomes the digital asset. Back-endservices, such as the user-experience server 702, associate digitalassets with unique identifiers. The unique identifier is then stored inassociation with a reference to the digital asset. The uniqueidentifiers may be used to retrieve the digital assets and output thoseassets to a recipient of the greeting card.

Turning now to FIG. 4, the recipient of the greeting card 100 opens thegreeting card application in the recipient's mobile device 130 and scansthe greeting card. As before, the greeting card application may provideinstructions to the recipient to scan an adequate portion of the visualmedia. The adequate portion should include the signature 110 and theminimum required fiducial marks needed to reproduce the uniqueidentifier. The adequate portion may include the SKU 105. If not allrequired fiducial marks are within the field of view, then the recipientmay be prompted to adjust the camera location.

Turning now to FIG. 5, the greeting card application then uses the imagecaptured by the recipient to reproduce the unique identifier using thecaptured signature and fiduciary marks. The unique identifier is thensent to the user-experience server with instructions to retrieve anydigital asset associated with unique identifier. The digital asset 132is then provided to the recipient's mobile device 130 and output to therecipient.

Turning now to FIG. 6, a block diagram is provided showing an operatingenvironment 600 in which some embodiments of the present disclosure maybe employed. It should be understood that this and other arrangementsdescribed herein are set forth only as examples. Other arrangements andelements (e.g., machines, interfaces, functions, orders, and groupingsof functions, etc.) can be used in addition to or instead of thoseshown, and some elements may be omitted altogether for the sake ofclarity. Further, many of the elements described herein are functionalentities that may be implemented as discrete or distributed componentsor in conjunction with other components, and in any suitable combinationand location. Various functions described herein as being performed byone or more entities may be carried out by hardware, firmware, and/orsoftware. For instance, some functions may be carried out by a processorexecuting instructions stored in memory.

Among other components not shown, operating environment 600 includes anumber of user devices, such as user devices 602 a and 602 b through 602n; a number of data sources, such as data sources 604 a and 604 bthrough 604 n; server 606; user 603, and network 610. It should beunderstood that environment 600 shown in FIG. 6 is an example of onesuitable operating environment. Each of the components shown in FIG. 6may be implemented via any type of computing device, such as computingdevice 1100, described in connection to FIG. 11. These components maycommunicate with each other via network 610, which may include, withoutlimitation, one or more local area networks (LANs) and/or wide areanetworks (WANs). In exemplary implementations, network 610 comprises theInternet and/or a cellular network, amongst any of a variety of possiblepublic and/or private networks.

It should be understood that any number of user devices, servers, users,and data sources may be employed within operating environment 600 withinthe scope of the present disclosure. Each may comprise a single deviceor multiple devices cooperating in a distributed environment. Forinstance, server 606 maybe provided via multiple devices arranged in adistributed environment that collectively provide the functionalitydescribed herein. Additionally, other components not shown may also beincluded within the distributed environment.

User devices 602 a and 602 b through 602 n may comprise any type ofcomputing device capable of use by a user 603. For example, in oneembodiment, user devices 602 a through 602 n may be the type ofcomputing device described in relation to FIG. 11 herein. By way ofexample and not limitation, a user device may be embodied as a personalcomputer (PC), a laptop computer, a mobile device, a smartphone, atablet computer, a smart watch, a wearable computer, virtual realityheadset, augmented reality headset, a personal digital assistant (PDA),an MP3 player, global positioning system (GPS) or device, video player,handheld communications device, gaming device or system, entertainmentsystem, vehicle computer system, embedded system controller, a camera,remote control, a bar code scanner, a computerized measuring device,appliance, consumer electronic device, a workstation, or any combinationof these delineated devices, or any other suitable device.

User devices 602 a and 602 b through 602 n can be client devices on theclient-side of operating environment 600, while server 606 can be on theserver-side of operating environment 600. Server 606 can compriseserver-side software designed to work in conjunction with client-sidesoftware on user devices 602 a and 602 b through 602 n to implement anycombination of the features and functionalities discussed in the presentdisclosure. This division of operating environment 600 is provided toillustrate one example of a suitable environment, and there is norequirement for each implementation that any combination of server 606and user devices 602 a and 602 b through 602 n remain as separateentities. Each user device 602 a and 602 b through 602 n can beassociated with one or more users, such as user 603. The users caninclude sender and recipients as described herein. Some user devices canbe associated with multiple users, such as a family PC, game console,meeting room PC, electronic white board, and such. Similarly, a singleuser can be associated with multiple devices, including shared devices.A user sign in identification can be used to determine the useroperating a user device at a point in time and to associate actionstaken with a user record.

Data sources 604 a and 604 b through 604 n may comprise data sourcesand/or data systems, which are configured to make data available to anyof the various constituents of operating environment 600, ormedia-experience system 700 described in connection to FIG. 7. Forinstance, in one embodiment, one or more data sources 604 a through 604n provide (or make available for accessing) digital assets that can beassociated with unique identifiers. The digital assets can be uploadedfrom user devices 602 a and 602 b through 602 n and/or from server 606.

Turning now to FIG. 7, a block diagram is provided illustrating anexemplary media-experience system 700 in which some embodiments of thepresent disclosure may be employed. The media-experience system 700includes network 610, which is described in connection to FIG. 6, andwhich communicatively couples components of media-experience system 700.The components of media-experience system 700 may be embodied as a setof compiled computer instructions or functions, program modules,computer software services, or an arrangement of processes carried outon one or more computer systems, such as computing device 1100 describedin connection to FIG. 11.

In one embodiment, the functions performed by components ofmedia-experience system 700 are associated with one or moreapplications, services, or routines. In particular, such applications,services, or routines may operate on one or more user devices (such asuser device 602 a), servers (such as server 606), may be distributedacross one or more user devices and servers, or be implemented in thecloud. Moreover, in some embodiments these components ofmedia-experience system 700 may be distributed across a network,including one or more servers (such as server 606) and client devices(such as user device 602 a), in the cloud, or may reside on a userdevice such as user device 602 a. Moreover, these components, functionsperformed by these components, or services carried out by thesecomponents may be implemented at appropriate abstraction layer(s) suchas the operating system layer, application layer, hardware layer, etc.,of the computing system(s). Alternatively, or in addition, thefunctionality of these components and/or the embodiments of thedisclosure described herein can be performed, at least in part, by oneor more hardware logic components. For example, and without limitation,illustrative types of hardware logic components that can be used includeField-programmable Gate Arrays (FPGAs), Application-specific IntegratedCircuits (ASICs), Application-specific Standard Products (ASSPs),System-on-a-chip systems (SOCs), Complex Programmable Logic Devices(CPLDs), etc. Additionally, although functionality is described hereinwith regard to specific components shown in media-experience system 700,it is contemplated that in some embodiments functionality of thesecomponents can be shared or distributed across other components.

The media-experience system 700 generally operates to identify a uniqueidentifier on a visual media, such as a greeting card, using a user'shandwritten marking, such as a signature, in combination with fiducialmarks pre-printed on the visual media. The unique identifier is used toassociate a digital asset, such as a video, with the greeting card. Thedigital asset can be retrieved when the recipient of the card scans thehandwritten marking and fiducial marks. As briefly mentioned above, eachcomponent of the media-experience system 700, including user-experienceserver 702, first user device 720, second user device 730, and storage790 and their respective subcomponents, may reside on a computing device(or devices).

The user-experience server 702 is responsible for providing theserver-side of the user experience. Using the greeting card as anexample, the user-experience server 702 will receive a unique identifierand a designated digital asset from the sender's user device. For thepurpose of illustration, the first user device 720 is described as thesender's user device and the second user device 730 is described as therecipient's user device. The user-experience server 702 will associatethe unique identifier and the digital asset together within a datastore. When the unique identifier is provided to the user-experienceserver 702 a second time, then the digital asset can be retrieved andcommunicated to the computing device that provided the unique identifierthe second time. Optionally an address, such as a URL, where a digitalasset can be found can be communicated instead of the digital assetitself.

The user-experience server 702 includes sub-components, such as digitalasset management component 712, handwritten marking manager 714, imagefeature extractor 713, handwritten marking ID component 715, andhandwritten marking storage 717. These components work together toprovide the experience described above.

The digital asset management component 712 manages digital assets andthe association of the digital asset with a unique identifier. Managingthe digital asset can include storing the digital asset. For example,pictures, videos, and audio recording provided by the sender may bestored by the digital asset management component 712. The digital assetsprovided by a particular sender could be stored in storage 790 andassociated with a user profile 740. In the example shown, the mediapairs 748 include a digital asset 748 a and the unique identifier 748 bassociated with it. A particular user, such as the sender, may beassociated with multiple media pairs. In instances where the digitalasset is already stored in association with a video provider, such as asocial media platform, the digital asset 748 a could be a link orreference to the digital asset. Any method of storing or identifying thedigital asset in such a way that it may be retrieved in the future aresuitable for use with the technology described herein.

Storing the media pairs in association with the user profile 740 canallow the user to modify the association. For example, a user thatinitially established the pairing may choose to delete the pairing,associate a different digital asset with the unique identifier, orotherwise update the pair.

The handwritten marking manager 714 receives images from a user deviceand generates a unique identifier that can be associated with thedigital asset. In some aspects, the handwritten marking manager 714 andits subcomponents could be located on a user device. The handwrittenmarking manager 714 can receive images from both the sender's device andthe recipient's device. Similar processes can be performed in eithercase. In the case of the sender, the image can be received as a uniqueidentifier. In the case of the recipient, the image can be received asecond time and the unique identifier generated and compared to existingidentifiers in the system.

The image feature extractor 713 can extract features from a signature orother marking received. Features extracted from the image can be used toindex the unique identifiers and can become part of the uniqueidentifier. The image feature extractor 713 identifies the location ofthe handwritten marking on the visual media using the fiducial marks.The location can be identified as a center point of the handwrittenmarking. The location also could be identified as a distance from afiducial mark. For example, a distance between the leftmost pixelincluding a portion of the signature and a fiducial mark could be onedistance. A second distance could be the distance between the right mostpixel and a fiducial mark. Multiple distances can be calculated in thismanner between various aspects of the signature or other marking andvarious fiducial marks within the image. Collectively, the plurality ofdistances calculated could define the handwritten marking location.

In one aspect, four or more points on the handwritten marking areidentified. For example, the rightmost, leftmost, topmost, and lowestpixels that include a portion of the handwritten marking may beidentified. If the handwritten marking image was converted toblack-and-white, with black pixels representing the ink, theabove-referenced pixels could be identified by analyzing an area of theimage associated with the handwritten marking and isolating blackpixels. A distance calculation for each of these identified pixels toone or more fiducial markers can be made. Thus, if two fiducial markersare used along with the four pixels mentioned, then eight distancecalculations could be made. These distance calculations could becombined in a hash function or some other manner to create a singlenumber that represents the location of the handwritten marking. Thesingle number could serve as part of the unique identifier. While it ispossible that multiple handwritten markings could result in the samesingle number, the single number is only one part of the uniqueidentifier. The other part comprises an image of the handwrittenmarking. Thus, the single number could be used to select a small subsetof stored unique identifier for further analysis in response toreceiving an image from a recipient.

Other features can also be identified. For example, the color of the inkused to make the handwritten marking can be a feature. The angle atwhich the handwritten marking, such as a signature, was made can beanother feature. Similarly, the height and length of the handwrittenmarking are features. Not all signatures are easily converted toletters, but a feature can include one or more letters extracted fromthe signature along with a confidence that the letter was correctlyidentified. Extraction of other features is possible.

The handwritten marking identification component 715 generates theunique identifier. In its simplest form, the unique identifier is theimage itself. In another aspect, the unique identifier is the imageafter undergoing some preprocessing, such as de-noising. In oneinstance, the image can be converted to a black-and-white image. Thefiducial marks can be used to scale the handwritten marking to a commonscale used to compare all handwritten marking. The unique identifier canalso include location characteristics of the signature. As previouslymentioned, the unique identifier could include a SKU.

The handwritten marking identification component 715 can also comparethe handwritten marking image in a first unique ID stored on the systemto the handwritten marking image provided by a recipient of a greetingcard or other visual media. In one aspect, a pixel-by-pixel comparisonis made of the original signature image to the second image. When thepixel-by-pixel comparison reveals a match, then the handwritten markingID component 715 can provide a match message to a requesting componentsuch as the digital asset management component 712.

The handwritten marking storage 717 stores the unique identifierincluding an image of the handwritten marking and associated fiducialmarks. As mentioned, the handwritten marking storage 717 may use one ormore aspects of the image to create a single identifier or number thatcan be used to retrieve the handwritten marking image for furtheranalysis. For example, a hash of handwritten marking location data maybe used by the handwritten marking storage 717 as an initial filter tofind unique identifiers requiring further analysis to confirm a match.In one aspect, if the single number associated with a unique identifieris unique the no further analysis is required of the image to confirm amatch.

The first user device 720 and the second user device 730 are provided.As mentioned, the first user device 720 can correspond to a user deviceused by a sender, while the second user device can be used by recipient.Both devices can include exactly the same components, but thesecomponents may perform different functions depending on whether thedevice is a sender or recipient.

The user-experience application 722 can perform functions on behalf ofthe sender and receiver. The functions performed by the user-experienceapplication 722 working on behalf of the sender are explained inconjunction with the user-experience application 722. Functionsperformed by a recipient are explained subsequently in association withthe description of the user-experience application 732. Theuser-experience application 722 could be a cloud-based experienceprovided through a web browser. The user-experience application 722could be a specialized application that runs on the first user device720 and communicates with a back end service.

The user-experience application 722 can generate themedia-identification interface 726. The media-identification interface726 can provide instructions asking the user to sign or otherwise add ahandwritten marking to the visual media. Further instructions to capturean image of the marked visual media and associated fiducial marks can beprovided. The instructions can use a combination of textual explanationand images.

The image capture component 724 captures an image of the handwrittenmarking, fiducial marks, and optionally an SKU. The image capturecomponent 724 can then send the image to the user-experience application722 for further use.

The digital asset collection 725 can provide an interface that allowsthe user to select a digital asset to associate with the visual media.The digital assets can include assets located on the mobile device, suchas existing pictures and videos. The digital assets can include assetslocated on a server, such as publicly or privately available picturesand videos. The digital asset may also be created by themedia-identification interface. For example, the media-identificationinterface 726 can facilitate the user recording an audio or videomessage that becomes the digital asset. The unique identifier and thedigital asset and/or identification information for the digital assetare communicated to the user-experience server 702. The user-experienceserver 702 maintains an association between digital assets and uniqueidentifiers. The unique identifiers may be used to retrieve the digitalassets and output those assets to a recipient of the greeting card.

The second user device 730 is used by the recipient of the visual media.For example, a sender associated with the first user device 720 couldmail a greeting card to the user of the second user device 730. Thesecond user could then open the user-experience application 732 andfollow instructions provided to retrieve a digital asset associated withthe visual media by the first user. The user-experience application 732could be a cloud-based experience provided through a web browser. Theuser-experience application 732 could be a specialized application thatruns on the second user device 730 and communicates with a back endservice. The user-experience application 732 can generate amedia-identification interface 736 that instructs the user to take apicture of the handwritten marking and associated fiducial marks. Theimage capture component 734 can capture an image of the handwrittenmarking and fiducial marks and communicate the image to theuser-experience application 732. The image can then be communicated tothe user-experience server 702. The user-experience server 702 analyzesthe image and identifies the digital asset associated with the uniqueidentifier generated by analysis of the image. The digital assetcollection 735 can perform functions that allow the digital asset to beoutput to the user through the second user device 730.

User profile 740 includes user accounts and activity data 742, userdevice information 744, user preferences 746, and media pairs 748. Useraccount(s) and activity data 742 generally includes user data related tothe sending and receiving of visual media using the user-experienceserver. In particular, user account(s) and activity data 742 can includedata regarding user emails, texts, instant messages, calls, and othercommunications; social network accounts and data, such as news feeds;online activity; calendars, appointments, or other user data used bycomponents of the system. Embodiments of user account(s) and activitydata 742 may store information across one or more databases, knowledgegraphs, or data structures. As discussed hereinabove, user deviceinformation 744 may include identification information for devices usedby a single user. The user device information can be used for securityreasons to validate a user, maintain a secure session, and for otherpurposes. User preferences 746 can include various permissions, such ashow long digital assets and the association between the digital assetsand the unique identifier should be maintained. The user preferences caninclude opt-in and opt-out selections for the sharing and collection ofpersonal information.

Turning now to FIG. 8, a method 800 a method for uniquely identifying avisual media is provided. Method 800 can be performed by a user device,such as user device 720.

At step 810, a first image of a portion of the visual media is obtainedfrom a first user device. The portion includes a handwritten marking ofa user and a plurality of fiducial marks. The handwritten marking couldbe a signature. A media-identification interface may provideinstructions to the user to capture the portion of the greeting cardincluding the handwritten marking and the minimum required amount offiducial marks. In other words, the portion of the visual media capturedby the camera should include all fiducial marks required to generate aunique identifier that includes the handwritten marking. For example, itmay be sufficient to include only two fiducial marks in the image inorder to create a unique identifier. However, when the SKU is includedon the visual media the instruction may request that the SKU also beincluded in the image. In this case, instructions could indicate thatthe handwritten marking should be captured along with at least twofiducial marks and the SKU.

The media-identification interface and/or greeting card application canperform a quality check on the image. If all required fiducial marks andthe SKU watermark are not within the captured image, then the sender maybe prompted to capture a second image. Alternatively or additionally,the quality check can be performed in real time through analysis of thevideo being received through the camera and instructions provided toadjust the camera location as needed. If all fiducial marks and SKUwatermark are within the field of view, then the greeting cardapplication instructs the user to capture an image, for example bytapping the screen.

At step 820, an instruction to associate the visual media with a digitalasset is received.

At step 830, the first image and the digital asset are associatedtogether within a data store.

Turning now to FIG. 9, a method 900 a method for uniquely identifying avisual media is provided. Method 900 may be performed by second userdevice 730.

At step 910, a media-identification interface is output through a userdevice that includes an image capture control that shows an image beingcaptured by a camera on the user device and an instruction portiondirecting a user to point the camera at a portion of the visual media.The instructions can include text and images. In other words, theportion of the visual media captured by the camera should include allfiducial marks required to generate a unique identifier that includesthe handwritten marking, such as a signature. For example, it may besufficient to include only two fiducial marks in the image in order tocreate a unique identifier. However, when the SKU is included on thevisual media the instruction may request that the SKU also be includedin the image. In this case, instructions could indicate that thehandwritten marking should be captured along with at least two fiducialmarks and the SKU.

At step 920, a user instruction is received through themedia-identification interface to capture a first image of the portionof the visual media. The portion includes a handwritten marking of auser and a plurality of fiducial marks. The media-identificationinterface can perform a quality check on the image. If all requiredfiducial marks and the SKU watermark are not within the captured image,then the sender may be prompted to capture a second image. Alternativelyor additionally, the quality check can be performed in real time throughanalysis of the video being received through the camera and instructionsprovided to adjust the camera location as needed. If all fiducial marksand SKU watermark are within the field of view, then themedia-identification application instructs the user to capture an image,for example by tapping the screen.

At step 930, the first image and an instruction to retrieve a digitalasset are communicated to a user-experience server. The digital asset isidentified by analyzing the handwritten marking and a location of thehandwritten marking in relation to the plurality of fiducial marks, asdescribed previously with reference to FIG. 7.

At step 940, the digital asset is received from the user-experienceserver.

At step 950, the digital asset is output through the user device.

Turning now to FIG. 10, a method 1000 for uniquely identifying a visualmedia is provided.

At step 1010, a media-identification interface is output through a userdevice that includes an image capture control that shows an image beingcaptured by a camera on the user device and an instruction portiondirecting a user to point the camera at a portion of the visual media.

At step 1020, a user instruction is received through the user device tocapture a first image of the portion of the visual media. The portionincludes a handwritten marking of a user and a plurality of fiducialmarks.

At step 1030, a user input is received through the user device toassociate the visual media with a digital asset.

At step 1040, an instruction is communicated to a user-experience serverto associate the image and the digital asset together.

Exemplary Computing Environment

With reference to FIG. 11, computing device 1100 includes a bus 1110that directly or indirectly couples the following devices: memory 1112,one or more processors 1114, one or more presentation components 1116,one or more input/output (I/O) ports 1118, one or more I/O components1120, and an illustrative power supply 1122. Bus 1110 represents whatmay be one or more busses (such as an address bus, data bus, orcombination thereof). Although the various blocks of FIG. 11 are shownwith lines for the sake of clarity, in reality, these blocks representlogical, not necessarily actual, components. For example, one mayconsider a presentation component such as a display device to be an I/Ocomponent. Also, processors have memory. The inventors hereof recognizethat such is the nature of the art and reiterate that the diagram ofFIG. 11 is merely illustrative of an exemplary computing device that canbe used in connection with one or more aspects of the presenttechnology. Distinction is not made between such categories as“workstation,” “server,” “laptop,” “handheld device,” etc., as all arecontemplated within the scope of FIG. 11 and with reference to“computing device.”

Computing device 1100 typically includes a variety of computer-readablemedia. Computer-readable media can be any available media that can beaccessed by computing device 1100 and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable media may comprisecomputer-storage media and communication media.

Computer-storage media includes both volatile and nonvolatile, removableand non-removable media implemented in any method or technology forstorage of information such as computer-readable instructions, datastructures, program modules, or other data. Computer storage mediaincludes, but is not limited to, RAM, ROM, EEPROM, flash memory or othermemory technology, CD-ROM, digital versatile disks (DVDs) or otheroptical disk storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information and which can be accessed bycomputing device 1100. Computer storage media does not comprise signalsper se.

Communication media typically embodies computer-readable instructions,data structures, program modules, or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media, such as awired network or direct-wired connection, and wireless media, such asacoustic, RF, infrared, and other wireless media. Combinations of any ofthe above should also be included within the scope of computer-readablemedia.

Memory 1112 includes computer storage media in the form of volatileand/or nonvolatile memory. The memory may be removable, non-removable,or a combination thereof. Exemplary hardware devices include solid-statememory, hard drives, optical-disc drives, etc. Computing device 1100includes one or more processors 1114 that read data from variousentities such as memory 1112 or I/O components 1120. Presentationcomponent(s) 1116 presents data indications to a user or other device.Exemplary presentation components include a display device, speaker,printing component, vibrating component, and the like.

The I/O ports 1118 allow computing device 1100 to be logically coupledto other devices, including I/O components 1120, some of which may bebuilt in. Illustrative components include a microphone, joystick, gamepad, satellite dish, scanner, printer, wireless device, etc.

The I/O components 1120 may provide a natural user interface (NUI) thatprocesses air gestures, voice, or other physiological inputs generatedby a user. In some instances, inputs may be transmitted to anappropriate network element for further processing. An NUI may implementany combination of speech recognition, touch and stylus recognition,facial recognition, biometric recognition, gesture recognition both onscreen and adjacent to the screen, air gestures, head and eye tracking,and touch recognition associated with displays on the computing device1100. The computing device 1100 may be equipped with depth cameras, suchas stereoscopic camera systems, infrared camera systems, RGB camerasystems, and combinations of these, for gesture detection andrecognition. Additionally, the computing device 1100 may be equippedwith accelerometers or gyroscopes that enable detection of motion. Theoutput of the accelerometers or gyroscopes may be provided to thedisplay of the computing device 1100 to render immersive augmentedreality or virtual reality.

Some aspects of computing device 1100 may include one or more radio(s)1124 (or similar wireless communication components). The radio 1124transmits and receives radio or wireless communications. The computingdevice 1100 may be a wireless terminal adapted to receive communicationsand media over various wireless networks. Computing device 1100 maycommunicate via wireless protocols, such as code division multipleaccess (“CDMA”), global system for mobiles (“GSM”), or time divisionmultiple access (“TDMA”), as well as others, to communicate with otherdevices. The radio communications may be a short-range connection, along-range connection, or a combination of both a short-range and along-range wireless telecommunications connection. When we refer to“short” and “long” types of connections, we do not mean to refer to thespatial relation between two devices. Instead, we are generallyreferring to short range and long range as different categories, ortypes, of connections (i.e., a primary connection and a secondaryconnection). A short-range connection may include, by way of example andnot limitation, a Wi-Fi® connection to a device (e.g., mobile hotspot)that provides access to a wireless communications network, such as aWLAN connection using the 802.11 protocol; a Bluetooth connection toanother computing device is a second example of a short-rangeconnection, or a near-field communication connection. A long-rangeconnection may include a connection using, by way of example and notlimitation, one or more of CDMA, GPRS, GSM, TDMA, and 802.16 protocols.

Many different arrangements of the various components depicted, as wellas components not shown, are possible without departing from the scopeof the claims below. Embodiments of the present disclosure have beendescribed with the intent to be illustrative rather than restrictive.Alternative embodiments will become apparent to readers of thisdisclosure after and because of reading it. Alternative means ofimplementing the aforementioned can be completed without departing fromthe scope of the claims below. Certain features and sub-combinations areof utility, may be employed without reference to other features andsub-combinations, and are contemplated within the scope of the claims.

1. A method for uniquely identifying a greeting card, comprising:obtaining from a first user device a first image of a portion of thegreeting card, wherein the portion includes a handwritten marking of auser and a plurality of fiducial marks, wherein the plurality offiducial marks are integrated with an artistic design on the greetingcard; receiving an instruction to associate the greeting card with adigital asset; and associating the first image and the digital assettogether within a data store.
 2. The method of claim 1, wherein themethod further comprises: receiving a second image of the portion of thegreeting card from a second user device; identifying the digital assetby comparing the handwritten marking and a location of the handwrittenmarking relative to the plurality of fiducial marks in the first imageto the handwritten marking and the location of the handwritten markingrelative to the plurality of fiducial marks in the second image; andcommunicating the digital asset to the second user device.
 3. The methodof claim 2, wherein the first image is obtained from a first instance ofa specialized application running on the first user device and thesecond image is obtained from a second instance of the specializedapplication running on the second user device.
 4. The method of claim 3,wherein the specialized application provides instructions for scanningthe portion of the greeting card.
 5. The method of claim 3, wherein thespecialized application performs an analysis to confirm that the portioncomprises all fiducial marks printed on the greeting card.
 6. The methodof claim 1, wherein the portion includes a printed watermark thatidentifies a product class of the greeting card.
 7. (canceled)
 8. Amethod for uniquely identifying a visual media, comprising: outputting amedia-identification interface through a user device that includes animage capture control that shows an image being captured by a camera onthe user device and an instruction portion directing a user to point thecamera at a portion of the visual media; receiving a user instructionthrough the media-identification interface to capture a first image ofthe portion of the visual media, wherein the portion includes ahandwritten marking of a user and a plurality of fiducial marks;communicating, to a user-experience server, the first image and aninstruction to retrieve a digital asset that is identified by analyzingthe handwritten marking and a location of the handwritten marking inrelation to the plurality of fiducial marks; receiving the digital assetfrom the user-experience server; and outputting the digital assetthrough the user device.
 9. The method of claim 8, wherein the firstimage is obtained from a first instance of a specialized applicationrunning on the user device.
 10. The method of claim 8, wherein theportion includes a printed watermark that identifies a product class ofthe visual media, and wherein the printed watermark is used with thelocation and the handwritten marking to identify the digital asset. 11.The method of claim 8, wherein the media-identification interface isprovided by a first instance of specialized application running on theuser device.
 12. The method of claim 8, wherein the method comprisesperforming an analysis of the image to confirm that the portioncomprises all fiducial marks printed on the visual media.
 13. The methodof claim 12, wherein the method comprises providing a new instruction tothe user to capture a second image when the portion does not compriseall fiducial marks printed on the visual media.
 14. The method of claim8, wherein the visual media is a greeting card.
 15. The method of claim14, wherein the digital asset is a video of a person that sent thegreeting card to the user of the user device.
 16. One or more computerstorage media having computer executable instructions embodied thereonthat, when executed by a computing device, causes the computing deviceto perform a method of uniquely identifying a visual media, the methodcomprising: outputting a media-identification interface through a userdevice that includes an image capture control that shows an image beingcaptured by a camera on the user device and an instruction portiondirecting a user to point the camera at a portion of the visual media;receiving a user instruction through the user device to capture a firstimage of the portion of the visual media, wherein the portion includes ahandwritten marking of a user and a plurality of fiducial marks;receiving a user instruction through the user device to capture a firstimage of the portion of the visual media, wherein the portion includes ahandwritten marking of a user and a plurality of fiducial marks;receiving a user input through the user device to associate the visualmedia with a digital asset; and communicating an instruction to auser-experience server to associate the image and the digital assettogether.
 17. The one or more computer storage media of claim 16,wherein the method further comprises outputting a digital assetselection interface that allows the user to select the digital assetfrom a plurality of existing digital assets.
 18. The one or morecomputer storage media of claim 16, wherein the method further comprisesoutputting a digital asset selection interface that allows the user tocreate the digital asset using a camera in the user device.
 19. The oneor more computer storage media of claim 16, wherein the method comprisesperforming an analysis of the image to confirm that the portioncomprises all fiducial marks printed on the visual media.
 20. The one ormore computer storage media of claim 16, wherein the method furthercomprises outputting a digital asset selection interface that allows theuser to create the digital asset using a microphone in the user device.21. A method for uniquely identifying a visual media, comprising:obtaining from a first user device a first image of a portion of thevisual media, wherein the portion includes a handwritten marking of auser and a plurality of fiducial marks, wherein the portion includes aprinted watermark that identifies a product class of the visual media;receiving an instruction to associate the visual media with a digitalasset; and associating the first image and the digital asset togetherwithin a data store.